Papers with deep representations
A Shared Geometry of Difficulty in Multilingual Language Models (2026.acl-short)
Copied to clipboard
| Challenge: | Large language models encode problem difficulty as an internal signal that can be linearly decoded from their residuals. |
| Approach: | They train linear probes on the AMC subset of the Easy2Hard benchmark, translated into 21 languages, and found difficulty-related signals emerge at two distinct stages of the model internals. |
| Outcome: | The results show that difficulty-related signals emerge at two distinct stages of the model internals, corresponding to shallow (early-layers) and deep (later-layer) representations, that exhibit functionally different behaviors. |
Correlation Coefficients and Semantic Textual Similarity (N19-1)
Copied to clipboard
| Challenge: | Existing research into semantic textual similarity has focused on word embeddings . little attention has been devoted to similarity measures between word embeds - a new study shows . |
| Approach: | They show that cosine similarity is essentially equivalent to the Pearson correlation coefficient for all common word vectors. |
| Outcome: | The proposed model outperforms the existing model on word-level and sentence-level similarity benchmarks. |
Modality Alignment between Deep Representations for Effective Video-and-Language Learning (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing Video-and-Language models do not take into account the different characteristics of video and text representations. |
| Approach: | They propose a method that exploits Centered Kernel Alignment (CKA) to enhance cross-modality attention by combining multiple modalities. |
| Outcome: | The proposed method outperforms conventional multi-modal methods significantly on video QA tasks with +3.57% accuracy increment compared to the baseline in a popular benchmark dataset. |
EMO: Embedding Model Distillation via Intra-Model Relation and Optimal Transport Alignments (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation focus on direct output alignment, neglecting this crucial structural information. |
| Approach: | They propose a framework for knowledge distillation that maps tokens one-to-one and aligns attention matrix patterns using Centered Kernel Alignment. |
| Outcome: | The proposed framework significantly outperforms existing CTKD baselines. |
Single-Pass, Depth-Selective Reading for Multi-Aspect Sentiment Analysis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models re-encode the sentence for each aspect or rely on static use of deep representations, leading to redundant computation and limited adaptivity. |
| Approach: | They propose a single-pass inference framework that encodes each sentence once to construct a reusable, depth-ordered substrate. |
| Outcome: | Experiments show that DABS reduces end-to-end computation by 60% in multi-aspect settings. |
Leveraging BERT and TFIDF Features for Short Text Clustering via Alignment-Promoting Co-Training (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing clustering methods rely on keyword information, but they lack this information. |
| Approach: | They propose a CO**-**T**raining **C**lustering framework to make use of BERT and TFIDF features. |
| Outcome: | The proposed framework outperforms existing SOTA methods on eight datasets. |
What’s in a prompt? Language models encode literary style in prompt embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models encode textual information using high-dimensional latent spaces . many studies have investigated how conceptual content of words translates into geometrical relationships between their vector representations . |
| Approach: | They use literary pieces to show that intangible, rather than factual, aspects of the prompt are contained in deep representations. |
| Outcome: | The results show that word-to-vec(tor) embeddings are more complex than other models. |